DOCUMENT RESUME 



ED 355 737 
TITLE 

INSTITUTION 

PUB DATE 
NOTE 

PUB TYPE 
JOURNAL CIT 

EDRS PRICE 
DESCRIPTORS 



EC 301 994 

Assessing the Transition Skills of Adolescents and 
Young Adults Who Are Deaf through Video 
Technology. 

Western Oregon State Coll*, Monmouth. Teaching 
Research Div. 
Dec 92 
13p. 

Collected Works - Serials (022) 
Teaching Research; Dec 1992 

MFOl/PCOl Plus Postage. 

Adolescents; Daily living Skills; '^Deafness; 
Diagnostic Tests ; Employment ; Federal Programs ; 
Higher Education; ^'Independent Living; '"^Job Skills; 
Screening Tests; Secondary Education; Standardized 
Tests; ''^Student Evaluation; '"'Test Construction; Test 
Format; Transitional Programs; '^Videodisks ; Young 
Adults 



ABSTRACT 

This newsletter issue describes a programmatic line 
of research to develop and standardize a test battery of transition 
skills (employment and independent living skills) for adolescents and 
young adults (ages 14 to 25) who are deaf. The newsletter first 
discusses the importance of assessment data to the transition process 
for persons who are deaf. Three federally funded research projects 
are then summarized, including: (1) development and testing of the 
Transition Comipetence Battery (TCB) , which involved specification of 
the content blueprint, generation of test items, pilot test of a 
written and signed (video) format, and analysis of test items and 
subtests; (2) examination of the small group administration mode and 
multiple-choice response format, through videodisc; and (3) 
development of a shorter instrument to u»e as a screening tool. The 
TCB comprises six subtests: job seeking skills, work adjustment 
skills, job-related social and interpersonal skills, money management 
skills, health and home skills, and community awareness skills. 
(Contains 28 references.) (JDD) 
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Special Education Activities 

The purpose of this newsletter is to share with you our activities and projects. Each issue features a different project or activity. 
This issue describes the development and standardization of a test battery for transition skills (employment and independent living 
skills). 



A list of our demonstration sites and those who manage them follows: 

Early Childhood/Special Education 

Teaching Research Child Development Center 
Director Pam Deardoiff 

Associate Teachers: Cassie Kioeker, Claudia Austin-Prcvost 
Assistant Teachers: Doris Maruame, Cindy Brown 
Early Intervention Staff: 

Coordiiuton Patty Scvems 

Infant/Toddler Specialists: Margaret McCaffrey, Muriel Liggett 



• PERMISSION TO REPRODUCE THIS 
MATERI/kL HAS BEEN GRANTED BY 



TO THE EDUCATIONAL RESOURCES 
INFORMATION CENTER (ERIC)/* 



Services for Persons with Moderate and Severe Disabilities 

Training Staff: Tony Templeman, Carol Bunse, Joyce Peters, Gary Glasenapp, Kathy Haydon, Lisa Carlson, Tom Udell 

Supported Work Training for Adults: Kirk Hendrickson, Miriam Waintrup, John Gallagher, Kim Miller 

Residential Supervisor Bud Fredericks 

Group Home for Adolescents with Severe Handicaps, Polk County: Eric Karlinger 

Group Home for Adults with Profound Handicaps, Clackamas County: Daphne Richardson 

Adult Sex Offender Program: Trish Mark 

Marion County Children's Project: Evelyn Fenis 



Services for Students with Severe Emotional Disturbances 
Supervisor Vicki Evans 

High School Resource Room, Salem Public Schools: TBA 
Training in Qassroom Services: Vicki Evans 

Vocational Program for Adolescents, Marion County: PhOiip Martinez 
Group Home for Adolescents, Marion County: Kelly Knechtel 
Apartments for Adolescents: Jacque Edinger 
Foster Parent Training: Vicki Evans 



ASSESSING THE TRANSITION SKILLS OF ADOLESCENTS AND YOUNG ADULTS 
WHO ARE DEAF THROUGH VIDEO TECHNOLOGY 



This issue of the Teaching Research Newsletter describes a 
programmatic line of research directed by Dr. Michael Bullis 
to develop and standardize a test battery of transition skills 
(employment and independent living skills) for adolescents and 
young adults (ages 14 to 25) who are deaf. In the sunmier of 
1986 Dr. Bullis was awarded a Field Initiated Research grant 
from the federal OfGcc of Special Education Programs (OSEP) 
to begin the development process of the Transition 

Nv. Competence Battery (TCB) (Reimaa & Bullis, 1990). More 
importantly, the notification of this award coincided to the day 
with the adoption of his son (Matt, or Giant as he is known to 
many). Shortly thereafter. Dr. John Reiman and Dr. Cheryl 

^ Davis were hired to coordiiute various aspects of the work. In 
1989 a second grant was awarded to Dr. Bullis by OSEP to 
^ explore the efGcacy of different administration and response 
formats for the TCB, and to transfer the TCB to a videodisc 
format. In the summer of 1992 a third grant was awarded to 
Dis. Reiman and Bullis from the National Institute on 
Disability and RehaNlitation Research's Field Initiated 

ERIC 



Research competition in order to develop and standardize and 
shortened version of the TCB, the "Mini-TCB." 

This newsletter first discusses the importance of assessment 
data to the transition process for persons who arc deaf. 
Following that section, each of the three projects are 
sununarized. 

Background 

Reliable and valid assessment data should be gathered to 
guide and structure focused and effective transition intervention 
programs for all persc/ns with disabilities (DeSte£ano, 1987; 
Frcy, 1984; Marut & Innes, 1986; Shiels, 1986; Sligar, 1983). 
Unfortunately, there are few instruments designed specifically 
to assess the transitic n skills of deaf adolescents and young 
adults (Reiman & ]3ullis, 1987). Typical practice is to 
administer traditional psychometric tests (e.g., IQ tests) or 
functional measures designed for other populations (e.g., 
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vocational skill tests developed for learning disabled 
adolescents). Such measures are routinely presented by 
administrators (interpreters or clinicians) possessing varying 
levels of sign language competence (Levine, 1974; Stewart, 
1986). Based on the lack of other assessment alternatives, this 
approach is logical, but the veracity of data gathered in this 
manner— and subsequent intervention decisions based on these 
results— are questionable for two primary reasons. 

First, deafness is a condition deOned by its unique 
expressive and receptive commuracation modalities that differ 
signiOcantly from those of our English-based bearing society. 
Many deaf people use a bona Ode language (American Sign 
Language or ASL) witL no structural relationship to English; 
which relies on vistial rather than auditory encoding and 
decoding; and which has a ruie-govemcd phonology, syntax, 
and morphology (Reiman & Bullis, 1989). These fundamental 
communicative differences warrant paramount consideration 
when conducting any assessment of deaf people's transition 
skills. Can the deaf person use an interpreter in an effective 
manner in a job interview? Does the individual know his or 
her legal rights when interacting with a policeman? Can the 
person formulate a strategy to communicate effectively with 
co-workers? Questions such as these are highly relevant to 
successful community adjtistment. Review of published 
research on measurement procedures with this population, 
however, reveals that tests designed for other populations do 
not address these crucial skills in any systematic way, nor have 
investigations delineated the particular skills and content 
necessary for deaf people to succeed in work and living 
endeavois (Bullis & Reiman, 1989; Reiman & Bullis, 1987). 

Second, any time the administration procedures of a 
standardized assessment tool are altered, the validity of the 
resulting assessment data must be questioned. Consider a 
measure of functional skill knowledge that was devised for use 
with and standardized on a group of adolescents other than 
those who are deaf (e.g., learning disabled). If that tool is 
administered using sign communication in place of verbal 
instructions, such substitution violates the standardization 
procedures of the measure and technically invalidates the tool 
(Gciweck & Ysseldyke, 1979; Standards, 1985; Yoshida & 
Friedman, 1986). Consequently, the resulting data are suspect 
due to the absence of psychometric standards of the measure 
for that type of application. 

Accordingly, there was a clear need to develop language- 
appropriate and psychometrically sound measures of transition 
skills for deaf persons. Three fundamental assumptions guided 
the research and development process that we followed. 

First, one of the major stumbling blocks in conducting 
research with or understanding investigations of this group, is 
that all too often deaf people are regarded as homogeneous. 
Quite the contrary, this population is highly heterogeneous, 
encompassing people with varying levels of auditory 
capabilities, linguistic skills, cognitive abilities, social skills, 
and emotional development. Consequently, the construction of 
an assessment battery, that is cor;r.nt relevant for a particular 
subgroup, requires the clear delineation of the segment of the 
population for which the instniment is to be used. Based on 
results of previous research (Bullis, 1985), the population for 
whom this instniment was constructed are described in the 
following way. This description parallels the "low 
functioning" term (or as we prefer, "low-achieving") that is 
often used to characterize a group of deaf people who do not 
attend 4-year colleges or succeed in work or living endeavors 
in the community, and for whom few services and little 
research is available (Bowe, 1988). 



Members of the subject population are devoid of a 
seriously complicating secondary disability, but some 
may present mild secondary conditions (e.g., wear 
glasses, heart murmur, etc.). Further, members of the 
subject population possess limited Ei^ish reading 
skills (e.g., read at approximately the third grade level). 
The subject population does not include persons who 
go on to a 4-year college or university, but may 
include persons who attend community college or 
vocational/technical training centers. More than likely, 
members of the sample are those peisons who either 
dropout of high school, seek employment immediately 
upon leaving high school, or go on to some type of 
rehabilitation or community-based training program. 
FinaUy, members of the subject population have little 
experience and/or training in employment and 
independent living skills. 

Second, the assessment process should begin with an 
examination of the individual's knowledge of requisite 
transition skills to work and live successfully in the 
community. Knowledge of how to behave is a necessary 
foundation of behavior (Bandura 1977), and studies suggest 
that knowledge of functional skills is correlated to actual skill 
performance for persons with mild cognitive impairments 
(Bullis & Foss, 1986; Landman, Irvin, & Halpem, 1980). 

Third, in order for assessment to be connected to transition 
instTL 'don or training, it is critical that the measurement tools 
that arc used represent the content of the particular domain of 
concern. Stated differently, measures of functional skills 
should be composed of items that adequately sample the 
knowledge and skills necessary for the deaf person to succeed 
in his or her transition from the school to the community. It 
follows that the parameters of transition, i.e., the work and 
independent living skills necessary for the target group of deaf 
persons to achieve success in community settings, must be 
clearly defined. 

First Project: Development and Testing of the TCB 

In line with the above assumptions, a domain sampling 
model of te5* construction (Nunnally, 1978) was adopted in the 
fiist project. In this approach, the content definitions of the 
employment and independent living domains for the tirget 
population were determined. These boundaries were then 
utilized as a blueprint for test construction, with test items 
generated within these parameteis. 

Speciflcation of the Content Blueprint 

A workshop was held with 18 professionals from the 
Northwest in the field of deafness to identify critical work and 
independent living skills for the target population. The 
workshop employed the Nominal Group Technique (NGT) 
(Delbccq, van de Ven, & Gtistafson, 1975) to answer two 
questions. 

1. What are the five most important employment-related 
skills for a member of the target population? 

2. What arc the five most important independent living- 
related skills for a member of the target population? 

The lists of skills generated by workshop participants were 
grouped into three employment related subdomains (Job 
Seeking Skills, Work Adjustment Skills, Job Related 
Social/Interpersonal Skills) and three subdomains related to 
independent living (Money Management, Health and Home, 



Community Awareness). These skill areas were then evaluated 
in a national survey of practitioners and leading "experts" in 
the field of deafness, a technique designed to insure broad 
geographical and social validity of the skills (Kazdin, 1977). 
Respondents were instructed to rate each skill on two 4-point 
Likert scabs in terms of importance (its importance to the 
ultimate employment or indepe^ident living success of members 
of the target population) and on presence (the percentage of 
persons in the target population possessing the competency). 
The idea was to identify those skills that were most important 
but demonstrated by the lowest percentage of the target 
population. 

A toul of 307 deaf and hearing service providers 
(representing residential and mainstream schools, community 
colleges, and rehabilitation programs) completed the survey 
(BuUis & Reiman, 1989). Analyses were conducted with this 
data set to identify the most critical transition competencies, 
from the perspective of professional opinion. To complement 
and modify the content definition drawn from both the NGT 
and the national survey, a literature review in this subject area 
was used to clarify, expand, and/or condense this listing 
fReiman & Bullis,' 1987). The six subdomains and their 
associated content areas are presented in Table 1 (Employment 
E)omain) and in Table 2 (Independent Living Domain). These 
six subdomains of transition for this part of the deaf population 
eventually became the six subtests of the TCB, and provided 
the framework used to structure the item generation activities. 

Generation of Test Items 

A second group of 20 professional service providers 
(hearing and deaf) were assembled for training on test item 
construction. These persons possessed direct experience with 
the target population in worK and/or independent living 
programs, and a working knowledge of sign communication 
(ASL, Pidgin Sign English, and Manually Coded English). 
The goal of the workshop was to teach the requisite skills for 
writing test questions in each of the previously identified 
content areas. 

Particular emphasis was placed on defining those categories 
in which TCB items would be wrinen-the knowledge, 
comprehension and application categories (Bloom, 1956)— in 
the form of 3-option, multiple choice questions. The format 
was chosen to minimize correct answers due to guessing 
(Nunnally, 1978) and because 3-option multiple choice 
questions have been demonstrated to be valid for use with 
adolescents and young adults with mild cognitive disabilities 
(Bullis & Foss,'l986: Landman et al., 1980). Linguistic 
structures (i.e., conditionals, minimal iiifomiation pronouns, 
comparative, negatives, etc.) of potential difficulty for the 
target population were next identified and their use cautioned. 
Lists ot words commensurate with a third grade reading level 
(the TCB target level) were introduced and reviewed. Finally, 
highly structured practice time for item writing was included 
with ail participants writing and critiquing practice sets of 
questions. 

Six weeks after the workshop, participants submitted more 
than 900 test items that were subsequently edited for content, 
duplication, and adherence to item construction criteria. 
Editing involved eliminating or significantly modifying 
inappropriate or inaccurate items and/or distractors. At the 
conclusion of the editing process, slightly more than 200 items 
remained that wers distributed across the six content 
subdomains and that addressed each of the content areas. 
Figure 1 presents sample items. 



Pilot Test of a Written and 
Signed (Video) Format 

To truly measure the subject population's knowledge of 
TCB conterlt-and not merely reflect their English language 
capabilities-a combined written and signed administration 
approach seemed appropriate. There are, though, few 
guidelines on which to base such a tool, so it was necessary to 
carefully examine the veracity of this admirustration approach. 
As a place to start, it was decided that a small group (n=4 to 
8) procedure that employed videotaped, signed directions 
coupled with a simply worded and illustrated test booklet, 
would be most expedient and cost effective. That is, a group 
of deaf persons would be shown the questions and their 
respective responses on the monitor and read the question in 
the test manual. On an individual basis they would mark the 
correct answer on a separate answer sheeL After a 
prespecified length of time the entire group would then be 
administered the next question. 

From the edited item pool, a subset of 30 test items v/as 
selected. These items were representative of multiple 
presentation styles (e.g. positioning and size of signer and 
character generation) and varying levels of reading complexity. 
(Note: Some items required students to read actual bus 
schedules or recipes as these are skills required in the "real 
world." In these instances the reading level of the items was 
not limited to the third grade level.) Correct answers and 
distractors for the 30 items were randomly positioned, and 
written materials including a test booklet and separate answer 
sheet v;:^^ developed. 

Next, a videotaped version of the 30 test items was 
developed in American Sign language (ASL), with a certified 
interpreter sigrung the question stem and the three possible 
responses. For many items, the salient information contained 
in the stem and/or the responses was reproduced using 
character generation which appeared simultaneously with, and 
just to the left of. the signer. 

Fhe pilot test was administered to 36 hearing impaired 
subjects located in three sites: mainstream high school 
jurdors/seniors, ii=8; residential high school seniors, n.= 16; 
community college deaf program students, n=:12. A process 
evaluation was conducted following each administration of the 
pilot test in the various sites. Overall, these comments were 
positive regarding the relevance of the test content and the 
level at which it was presented, but two very important points 
were apparent. First, the number of subjects reporting 
Jifficulty in understanding ASL raised serious questions 
regarding the viability of this language as the choice for the 
TCB. Second, subjects reported annoyance with the videotape 
moving too slowly. Both the length of the countdown and the 
response time between items were distressing for some 
students who, having answered the question, were ready to 
move to the l^.xl question without a prolonged waiting period. 
Each of these issues were carefully considered in the 
development of the TCB. 

Final Development 

From the results of the pilot-test it was decided that the 
videotaped, signed version of the TCB should be presented in 
Pidgin Signed English (PSE) (sign communication using 
primarily English word order with ASL signs and ASL 
grammatical features). The reported difficulties with the 
videotape moving too slowly necessitated a re-examination of 
the timing patten\s within and between test items. At issue 
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Table 1. Employment Skills and Content Areas 



Employment Domain 

Job Seeking Skills Subdomain 

E-1 The individual should be able to use an interpreter appropriately and effectively in a job interview. 
E-10 The individual should have knowledge of the pay and benefits that can be expected for the job. 
E-14 The individual should be aware of legal rights in getting a job and in terms of job advancement. 
E-16 The individual should display appropriate dress and hygiene when interviewing for a job. 

E-17 The individual should be aware of the language and terms that are used in the job application and in the interviewing 
process. 

E-18 The individual should be able to respond to and ask appropriate questions in the job interview. 
E-21 The individual should have knowledge of resources and agencies to use for help in finding a job. 
E-23 The individual should display appropriate assertiveness in searching for a job. 

E-24 The individual should be able to demonstrate appropriate resume writing skills and competence in completing job 
applications. 

E-2S The individual should demonstrate job-related reading skills 

Work Adjustment Subdomain 

E-3 The individual should be able to obtain training opportunities that are available through the current job. 

E-5 The individual should be able to manage a work schedule to meet deadlines. 

E-7 The individual should demonstrate appropriate dress and hygiene in the work place. 

E-8 The individual should be able to understand and follow the work supervisor's instructions. 

E-9 The individual demonstrate job-related mathematics skills. 

E-12 The individual should have knowledge of work safety rules. 

E-13 The individual should be able to deraonstrate appropriate procedures for quitting a job. 

E-20 The individual should be able to work without direct supervision when appropriate. 

£-25 The individual should to demonstrate job-related reading skills. 

Job^Related Social/Interpersonal Skills Subdomain 

E-2 The individual should be able to accept criticism from work supervisors. 

E-11 The individual should be able to communicate effectively with the work supervisor. 

E-19 The individual should display appropriate assertiveness on the job toward co-workeis. 

E-22 The individual should be able to keep personal concerns and worries under control in the work place. 

E-26 The individual should have an awareness of co-workers' and supervisors' lack of knowledge about dea&iess. 

E-27 The individual should demonstrate appropriate control and management of anger and frustration in the work place. 



Table 2. Independent Living and Content Areas 

Independent Living Domain 

Money Management Subdomain 

lL-1 The individual should demonstrate independent-living related mathematical skills. 

IL-2 The individual should be able to demonstrate effective comparative shopping skills (i.e., auTo, clothes). 

IL-3 The individual should demonstrate independent living reading skills. 

IL-8 The individual should be aware of appropriate skills in money management, budgeting, and bill paying. 

IL-21 The individual should be able to maintain accurate financial and personal records (i.e., taxes, warranties, medical). 

1X^25 The individual should have knowledge of how co itractual agreements work (e.g., rental contracts). 

Health & Home Subdomain 

IL-1 The individual should demonstrate independent-living mathematics skills. 

IL-3 The individual should demonstrate independent-living reading skills. 

IL-9 The individual should be able to search effectively for housing. 

IL-12 The individual should have knowledge of insurance needs (auto, health, life, home). 

IL-16 The individual should be able to maintain and care for pttsonal belongings. 

11^18 The individual should be able to access emergency services (e.g., ambulance) in the community. 

IL-20 The individual should have knowledge of family plaiming and sex education. 

IL-22 The individual should demonstrate appropriate cooking skills. 

IL-23 Tuc individual should demonstrate appropriate health related skills (nutrition, exercise, hygiene). 

lL-28 The individual should have knowledge of drug and alcohol abuse. 

Community Awareness Subdomain 

11^6 The individual should have knowledge of leisure/recrcationa) options in the community. 

11^7 The indiviriual should be able to communicate effectively with community workers (mail carriers, maintenance workers). 
Il^lO The individual should be able to use public transportation. 

11^15 The individual should be able to use TDD and technical signaling devices thai arc available in the community. 
IL-17 The individual should demonstrate appropriate self advocacy skills with community service agencies (i.e., DVR). 
IL-24 The individual should be aware of legal rights in the community. 
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3. Use the bus schedule above to answer this question. You live near Court and High. You work near Center and 24th. What time 
does the bus leave that will arrive at Center and 24th at 7:26? 

A. 7:15 a.m. 

B. 7:20 a.m. 

C. 7:45 a.m. 

28. You arc at a job interview with a boss and you are using an interpreter. Wherc should your interpreter sit during your job 
interview? 



Boss 



Desk 



Interpreter 



You 



Interpreter 
* Boss 



Desk 



•You 
• Bof>s 



Desk 



You 



• Interpreter 



(in the middle) 



(next to boss) 



(next to you) 
Figure 1. Sample Test Items 



was the need to allot increments of response time that would 
be reasonable given the heterogenous abilities of the target 
population. Based on staff observations during the pilot testing 
of the actual time individuals used to respond to each item, the 
timing patterns of the videotaped form were adjusted 
downward and gauged to respond to the middle on a 
continuum of subjects' demonstrated needs. Finally, the group 
administered, 3'K>ption multiple choice format for the ttst 
battery was maintained. 

All subtests were produced in a professional recording 
studio using a ceitified sign language interpreter and extensive 
use of character generation on the monitor to highlight key 
aspects of the test questions and their responses. Before 
beginning the testing, a set of practice items were administered 
on videotape to orient the individuals to the demands of the 
testing process. Items in each subtest were randomly assigned 
a position, and each item's responses (the correct response and 
the two distiactois) were randomly assigned to be either "a," 
"b," or "c." The six original subtests, with the amount of time 
required to administer each and their respective number of 
items, are listed below. 

* Subtest 1: Job Seeking Skills for Employment (54 
minutes:28 seconds; 38 items) 

O 
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* Subtest 2: Work Adjustment Skills for Employment 
(40:45-,32) 

* Subtest 3: Job Related Social/Interpeisonal Skills for 
Employment (34:38;27) 

^ Subtest 4: Money Management Skills for Independent 
Liidng (36:30;23) 

* Subtest 5: Health and Home Skills for Independent 
Uving (42:39-33) 

* Subtest 6: Community Awareness Skills for Independent 
Living (37:34;28) 

Field-testing of the TCB 

Field-testing of the TCB was conducted in 14 sites from 
across the country representing urban and rural settings, as well 
as residential and mainstream programs. In all cases, project 
staff visited each site, trained program staff in the 
administration of the TCB, and then either administered the 
subtests to all subjects or closely monitored the testing by 
program ^^isonnel. Because of the amount of time necessary 
to administer the entire battery, some sites administered only 
the employment subtests or only the independent living 
subtests. Thus, between 181 to 230 subjects participated in the 
Oeld-testing. Tbe majority were male (56% to 58%), from 
residential schools (53% to 74%), were deafened prelingually 
(before age 3) (79% to 84%), were between 18 to 19 yeais of 
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age at the time of testing (means 18.69 to 19.07), and read at 
around the third grade level (mean of 3.1 grade level). 

Item Statistics, Two different kinds of item statistics were 
computed for each subtest: item difiGculty, or the percentage of 
subjects who answered each item correctly; and point-biserial 
or item-total correlations, that indicate the relationship of each 
subtest item to the total subtest As a statistical guideline for 
retention in the TCB subtests, items were to possess a point- 
biserial correlation of at least 2 and a moderate level of 
difOculiy (between A to .8). Also to be retained, each item 
was judged to be conceptually appropriate. Consideration of 
these standards resulted in the deletion of 18 items. These 
decisions reduced the number of items in each subtest and the 
time required to administer each. 

* Subtest 1: Job Seeking Skills for Employment (48 
minutes: 13 seconds; 33 items) 

* Subtest 2: Work Adjustment Skills for Employment 
(39:48; 31) 

* Subtest 3: Job Related Social/Interpersonal Skills for 
Employment (33:35; 26) 

* Subtest 4. Money Management Skills for Independent 
Living (32:27; 20) 

* Subtest 5: Health and Home Skills for Independent 
Living (38:34; 29) 

* Subtest 6: Community Awareness Skills for Independent 
Living (33:12; 24) 

Subtest Characteristics. The average score of each subtest 
and its sta.adard deviation, and the average percent of the 
questions answered correctly for each subtest are presented in 
the two right columns of Table 3. 

Two types of reliability indices were computed for each of 
the TCB subiests. CoefOcient alpha internal consistency 
reliability indices are presented in the left column of Table 4. 
We also conducted a study of each subtest's test-retest 
reliability. Sixteen deaf persons representing mainstream and 
community college programs (9 males and 7 females, 13 of 
whom were deafened prc-lingually, with an average, age of 
20.387) were involved in the test-retest study of the 
employment subtests. Twenty-eight students from residential 
school programs (15 male and 13 female, 25 of whom were 
deafened prc-lingually, with an average age of 17.283) 
participated in the test-retest of the independent living subtests. 
The right column of Table 4 provides the test-retest reliability 
indices calculated for each subtest. In general, these indices 
are acceptable for a screening measure such as the TCB. 

In contrast to reliability, validity is a test property that must 
be established over time through repeated studies and in 
various ways (Messick, 1989; Nunnally, 1978). In this first 
project we were able to address two types of test validity: 
content and construct validity. 

Nunnally (1978) states that the content validity of a measure 
is demonstrated best through the procedures followed in its 
development. That is, the steps followed in identifying and 
sampling dictate in large part whether or not the test 
adequately samples content from the domain of concern. 
Given the extensive procedures followed to develop a content 
matrix across the employment and independent living domains, 
and to generate content-relevant test items, it can be judged 
that the TCB and its six subtests arc content valid. 

Construct validity, on the other hand, is demonstrated by the 
way the assessment instrument relates to a theoretical model of 
the constnict being measured (Keriinger, 1986; Messick, 1989; 



Nunnally, 1978). One method to establish constnict validity is 
the discriminant group or social comparison approach (Bellack 
8l Hersen, 1988; Bolton, 1987; Kerlinger, 1986; Nunnally, 
1978; Wiggins, 1973). In this technique, the researcher 
theorizes that a subject group for whom the particular measure 
was not constructed will perform in a very different way on 
the test than a subject group for whom the measure was 
designed. Results confirming the hypothesis provides evidence 
supporting the measure's construct validity. To establish such 
a criterion group, 13 deaf unde^raduate students were 
recmited to participate in this investigation who were 
administered all of the TCB subtests. It was hypothesized that 
this group would score differently, i.e., higher, on each of the 
six TCB subtests than would members of the tai^et population. 
To compare the college group's performance on the TCB with 
that of the target population's, two groups of residential 
subjects and two groups of mainstream subjects (each 
consisting of 13 subjects) were randomly selected from the 
standardization pool of subjects. Statistical comparisons were 
conducted on each of the subtests and these groups. These 
analyses resulted in highly statistically significant differences 
favoring the college student's performance on each of the 
subtests. Calculation of the effect size yielded results that 
would be considered as "large" differences (i.e., effect size 
greater than .80 between pairs of groups on a particular 
measure, Cohen [1988]). 

As a further test of the construct validity of the TCB, inter- 
correlations of the TCB subtests and pertinent demographic 
variables for the group of subjects ^= 158) that took the entire 
test battery were calculated. Statistically significant, but weak 
correlations (in the .20 range) were exhibited among gender, 
type of school program (maipstream vs. residential), age, and 
subtest performance. A pattern of low, negative correlations 
were found describing the relationship between pre- and 
postlingual hearing loss (hearing loss before age 3 as compared 
to hearing loss after age 3) and subtest performance. These 
results suggest that these demographic variables were not 
strongly related to test performance, providing additional 
evidence for this test property. 

Project 2: Examination of Administration Modes and 
Response Formats throu^ Videodisc 

While the results of the first project were strong, there was 
an aspect of the TCB that demanded further investigation. 
That is, the administration mode (small group, n=4 to 8) and 
response format (3-option multiple choice) chosen of the 
battery were based on "educated guesses," without benefit of 
any empirical guidance. In the fall of 1989, then, a second 
grant to study to address this issue was initiated. In this study 
the use of group, individual (a proctor administered the 
subtests to a single deaf person at a time), and self 
administration (the deaf person independently answered the test 
questions on his or her own by responding on a computer 
keyboard) combined with multiple choice and true/false 
response formats was examined. The videotape version of the 
TCB was transposed to videodisc, because of this technology's 
great advantages of flexibility and clarity, through the facilities 
at Utah State University under the direction of Dr. Ron 
Tborkildsen. 

Design/Production 

Different forms of two subtests of the TCB (one from the 
vocational domain and one from the independent living 
domain) were developed and transferred to interactive 
videodisc. The two subtests were Subtest 3- Job-Related 
Social Skills and Subtest 4- Money Management Skills. 



Table 3. Subtest CharacteiisUcs 



Subtest 1: Job Seeking Skills 

for Employment 
(n = 230) 

Subtest 2: Work Adjustment Skills 
for Employment 
(n = 181) 

Subtest 3: Job Related Social and 
Interpersonal Skills 
for Employment 
(n= 230) 

Subtest 4: Money Management Skills 
for Independent Living 
(n= 190) 

Subtest 5: Health and Home Skills 

for Independent Livir^ 
(n= 189) 

Subtest 6: Conmiunity Awareness Skills 
for Independent Living 
(11= 188) 



Mean Item 
Difficulty 
(mean, 
sd) 



.677 
.101 



.714 
.121 



.725 
.122 



.507 
.155 



^48 
.170 



.661 
.140 



Item-Total 
Correlation 
(mean, 
sd) 



374 
.122 



373 
.143 



.404 
.114 



.247 
.119 



.288 
.092 



338 
.126 



Mean 
Score 
(mean, 
sd) 



22348 
6.504 



22.144 
5.824 



18.843 
5.222 



10.132 
3.527 



15.884 
5.036 



15.851 
4.560 



Mean 
Percent 
Correct 



.677 



.714 



.725 



.507 



.548 



.661 



Number 
of Items 



33 



31 



26 



20 



29 



24 



Table 4. Subtest Reliabilities 
Internal 

Consistency Test-Retcst 

Subtest 1: Job Seeking Skills 

for Employment .864 .896 

Subtest 2: Work Adjustment Skills 

for Employment .857 .828 

Subtest 3: Job Related Social and 
Inteipeisonal Skills 

for Employment .860 .849 

Subtest 4: Money Management Skills 

for Independent Living ,668 .613 

Subtest 5: Health and Home Skills 

for Independent Living ,777 .766 

Subtest 6: Community Awareness Skills 

for Independent Living .801 .844 



The adaptation of the subtest forms confonned to the 
following sequence. Fiist, the original administration 
instructions for the TCB were rewritten to reflect the 
individually administered, self-administered, and true-false 
formats. The instructions for each form were essentially the 
same, with the exception of slight modiHcations related to 
content, response format, or administration mode. Second, for 
the appropriate form, test items were rewritten in the true/false 
response format Each item was randomly designated as being 
either true or false. If designated as true, the item was 
rewritten to embody the correct response altemative of the 
item. If designated as false, the incorrect distractor that drew 
the highest percentage of respondents in the Grst project was 
chosen and the item rewritten to reflect this response. Each 
administration mode utilized the same practice items, structured 
according to administration and response requirements of the 
particular experimental condition. 

The true/false forms were videotaped at facilities at Oregon 
State University in which the original TCB was produced. The 
tapes were edited and a master tape (1" master) was be sent to 
Utah State for transfer to videodisc. Each videodisc was 
restructured to contain a "clock bank." The clock bank 
contained the maximum amount of time that would ever be 
needed and through the branching nature of videodisc, only 
that portion of the total time per question would be utilized. 
The clock bank or timing screen included both the 
representation of a clock with hands and a digital clock to 
show remaining time, with a visual '^flash" near the end of the 
count down to get the attention of the user that another 
question was about to be presented. It was proposed that this 
flash occur at the seconds 5, 4, 3, 2, 1, and 0. 

Production for the Grst test disc was completed in the 
spring of 1990. After initial testing in Oregon, several minor 
problems were evident These issues were addressed by staflf 
at both institutions and resolved. The flnal versions of the 
subtests were produced in the fall of 1990, and prepared for 
Geld-testing. 

Method 

The purpose of this project was to examine the effect of 
different administration modes and response formats in the 
assessment of deaf adolescents and young adults. While it 
makes sense that different approaches and methods of 
assessment would produce different results, these differences 
must be judged in relation to some standard, or standards, of 
"correctness." For example, just because a particular 
administration mode and response format produced higher test 
performance than another, did not mean that these results were 
necessarily "better." Accordingly, the psychometric 
characteristics of the subtest and item characteristics were 
examined, as well as to study the relationship of test 
performance to an external standard. These criteria included 
third party ratings of actual skill and, in some cases, 
comparison of subtest performance to data from the 
preliminary standardization sample. 

Research Design. The design adopted for this project 
included cwo independent variables: administraticn mode 
(group, individual, and self) and item response format (multiple 
choice and true/false). It was not possible to randomly assign 
all subjects to experimental conditions, so sites were assigned 
to administration mode based on three pragmatic variables: 
entry into the project, number of subjects, and suted 
preference. Once the site was assigned to an administration 
condition, subjects within the site were assigned to a response 
format condition. That is, subjects were administered either 
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Subtest 3- Job-related Social Skills in the multiple choice 
response format and Subtest 4- Money Management Skills in 
the true/false response format, or Subtest 3- Job-related Social 
Skills in the true/false response format and Subtest 4- Money 
Management Skills in the multiple choice response format 
This research approach is best characterized as a two-way— 3 
(administration mode) x 2 (response format)— quasi- 
experimental design (Cook & Campbell, 1979) (Figure 2). 



Response 
Format 


Administration Mode 


GROUP 


INDIVIDUAL 


SELF 


MC 








T/F 









Key 

MC - Multiple Choice 
T/F - True/False 

Figure 2. Research Design 

Procedures^ A total of 309 deaf persons participated in 
this study. These persons represented 18 mainstream, 
residential, and rehabilitation sites horn across the country. At 
some sites Teaching Research staff administered the subtests, 
while at others program staff conducted the testing. In all 
cases, the administration procedures described previously were 
strictly followed. Third party rating scales were completed on 
most subjects at each site by third party raters who were 
judged as being sufficiently aware of each individual's work 
and independent living skills. 

Results^ At this writing ti>e final report for this project is 
being completed. Given space constraints it is not possible to 
present all of the findings, so key results are summarized 
below. 

* Item-total (Point-biserial) correlations for the total sample 
for Subtest 3- Job-related Social SkiiLs and Subtest 4- Money 
Management Skills favor the group and individual 
administration modes and the multiple choice response format 
In fact, the low point biserial correlations for the true/false 
response format indicate a more random test performance 
proGle in that condition as compared to the multiple choice 
response condition. 

* For both of the subtests, higher scores were associated 
with the true/false format and the individual and self 
administration modes. 

* Internal consistency reliability indexes were higher for 
the multiple choice response condition than for the true/false 
response coiKlition. However, the individual administration 
mode tended to result in a higher index than did the group 
administration mode. 

* For the entire sample, on both subtests, higher 
correlations were found between the multiple choice response 
condition and third party ratings, than between the true/false 
response condition and third party ratings. 

* Performance on the practice items (i.e., those who did 
not miss any practice items vs. those who did miss practice 
items) clearly was associated with varying performance levels 
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(those who did not miss practice items tended to score higher 
on both subtests). 

* The subgroup of subjects cvho did not miss any practice 
items exhibited higher internal consistency reliability indexes 
than those subjects who miss practice items. 

* The sut^oup of subjects who did not miss any practice 
items exhibited higher correlation between test scores and third 
party ratings across subtests than did other subjects. Within 
this group, the multiple choice response format demonstrated 
higher correlations than the true/false response condition. 

Taken together, these data suggest that the multiple choice 
response format was superior to the true/false response format 
exhibiting higher item statistics, internal consistency 
reliabilities, and correlations to outside criteria. Therefore, we 
believe it is safe to conclude that the multiple choice response 
format for the TCB is preferred. 

The issue of which administration mode optimum is not as 
c'.ear. Overall, it appears that the group administration mode 
is appropriate for most deaf adolescents and young adults 
(approximately 80% of those tested in this study) the group 
administration mode is appropriate and should be used, 
especially in light of its cost efficiency (e.g., being able to test 
a group of 4 to 8 subjects in the same time it takes to test 
one). However, this recommendation must be tempered by 
two points: (a) persons who missed practice items performed 
differently (i.e., poorer) than those who do not and (b) there is 
an indication that when the subtests were administered 
individually scores and psychometric characteristics were 
higher. Accordingly, it makes the most sense that to identify 
those subjects who miss any practice items and to assess them 
through an individual administration. There is an intuitive 
appeal to this recommendation, as well, as the one-to-one 
interaction provides the most control and direct information 
and impressions to the examiner. 

Two other points should be noted. First, the self 
administration mode was not fully tested in this study. From 
the subjects we did test, it appeared a viable approach. The 
subjects who were administered the subtests in this maimer, 
though, were older and in community college placements; thus, 
it may be that this imprescion would not hold for high school 
age, non-postsecondary bound deaf persons. Still, the self 
administration approach deserves further investigation. 
Second, Subtest 4- Money Management Skills remains an 
enigma, as performance on this subtest was variable and low. 
This Gnding, though, was consistent with the results of the 
initial project, so this result may not be indicative of any 
shortcoming in the subtest. That is, it may mean that this is an 
area that is inherently difficult and intensive instruction may be 
necessary for the majority of deaf adolescents and young adults 
to be protlcient in these skills. 

In summary, these results support the multiple choice 
response format and, for the majority of deaf individuals, the 
group administration format. The subgroup analyses revealed 
that persons who miss practice items tend to score lower than 
those who do not, and there was higher subtest performance 
when the measures were administered individually. 
Accordingly, we recommend that those persons who miss 
practice items should be identified and tested individually. 
Accordingly, all six of the TCB's subtests have been 
transposed to videodisc and the administration directions are 
being rewritten to reOect these findings* 



Project 3: Developmeiit of the Mini-TCB 

The response we have received to the TCB has been 
overwhelming positive. However, nearly all professionals who 
have participated in our projects state that they would prefer an 
instrument that is shorter to use as a screening tool. If the key 
items in the TCB reflecting overall performance could be 
identified, it would be possible to construct a shorter 
instrumeitt for this purpose. Performance on this measure 
could be used to identify gross skill deficits, and to develop a 
preliminary profile of the deaf subject's skill level across the 
six subtests. Areas identified as potentially weak, in need of 
further examinatiotu could be identified and assessed further 
with the appropriate subtest 

The purpose of this project, theiu will be to construct a 
screening instrument that could be used in just such a maimen 
The Mini-TCB. During this past year (1991-1992) we 
established a contract with Northern Illinois University and its 
Research and Training Center on Deafness, to conduct 
sophisticated item-parameter analyses. These procedures were 
conducted by Drs. Greg Long and Nisha Mittal and Ms. Chris 
Reid, to identify those items from the TCB that are the "best" 
predictors of subtest performance. At this pointy a pool of 50 
to 60 items have been identified and will be considered for 
inclusion in the Mini-TCB. In Year 1 of this project the initial 
version of the Mini-TCB will be produced on videodisc 
through the facilities at Utah State University, pilot-tested, and 
then finalized. In Year 2 the Mini-TCB will be field-tested in 
residential and mainstream schools across the country. In Year 
3, we will conduct two validation studies of the instrument 

Conclusion 

This issue concludes with two final thoughts. First It 
would be naive to believe that only one measure should be 
used in the appraisal of transition skills. Comprehensive 
assessment of deaf persons should include traditional 
psychometric measures, work evaluation samples, direct 
situational assessments, as well as the TCB. To identify such 
a battery, it will be necessary to conduct research to identify 
the most effective instruments to include such assessments. 

Second, in a very real sense, in the development of the 
TCB, we content-analyzed the specific type of work and living 
skills a deaf person in the target group must exhibit to succeed 
in transition. This information is critical for the development 
of an assessment battery, but also for training. Our next goal 
is to submit proposals that will allow us to develop curriculun; 
packages in each of the six content areas of the TCB. In this 
way, professionals would be offered an interlocking assessment 
and training program geared directly to this population. We 
are hopeful that at some point we will be able to write a 
newsletter detailing those fiiture projects. 
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MATERIALS LIST 

In the transition area the following materials are available 
from Teaching Research Publications. 

Associated work skills: A manual. The Teaching Research 
Special Education Department Staff. Teaching Research 
Publications, Monmouth, Oregon 97361. $10.00 

Functional Living Skills for Adolescents and Adults with Mild 
and Moderate Disabilities: Budgeting Skills. Nishioka- 
Evans, V., Kraus, D., Ferguson, C, & Fredericks, B. 
Teaching Research Publications, Monmouth, Oregon, 
97361. 1990. 

The Teaching Research curriculum for handicapped 
adolescents and adults: Personal hygiene. Fredericks, H. 
D., Makobon, L, Bunse, C, Heyer, M., Buckley, J., 
Alrick, G., & Samples, B. Teaching Research 
Publications, Monmouth, Oregon 97361. $10.00 

The Teaching Research curriculum for handicapj?ed 
adolescents and adults: Dressing, clothing care and 
selection. Fredericks, H. D., Heyer, M., Makohon, L ., 
Bunse, C, Buckley, J., Trecker, N., Egan, I., Johnson- 
Dom, N., Miller-Case, V., Fay, M. L., Paeth, M. A., 
Alrick, G., & Samples, B. Teaching Research 
Publications. Monmouth, Oregon 97361. $20.00 

The Teaching Research curiculum for handicapped 
adolescents and adults: Assessment procedures. Petersen, 
J., Trecker, N., Egan, I., Fredericks, H. D., & Bunse, C. 
Teaching Research Publications, Monmouth, Oregon 
97361. $10.00 

The Teaching Research curriculum for mildly and moderately 
handicapped adolescents and adults: Taxonomy and 
assessment. Nishioka-Evans, V., Hadden, C, Kraus, D., 
Johnson, J., Fredericks, H., & Toews, J. Teaching 
Research publications, Monmouth, Oregon. $10.00 

The Teaching Research curriculum for mildly and moderately 
handicapped adolescents and adults: Telephone skills. 
Nishioka-Evans, Fredericks, H., Toews, J., Hadden, C, 
Moore, W., and Dooley, M. Teaching Research 
Publications, Monmouth, Oregon. $10.00 

Transition for Persons with Deaf-Blindness and other Profound 
Handicaps. Fredericks, H. D., Covert, A. Teaching 
Research Publications, Monmouth, Oregon 97361. $10.00 

Vocational Training for Students with Severe Haitlicaps. H. 
D. Bud Fredericks and Staff of the Teachi/ug Research 
Vocational Training Model. Teaching Research 
Publications, Monmouth, Oregon 97361. $13.00 

To purchase the above, please fill out the order form and send 
to: 

i X 

10 



Teaching Research Publications 
345 N. Monmouth Avenue 
Monmouth, OR 97361 



Title(s) Oidcrcd; 



Name of Peison Ordering: 
Address: 



Purchase Order Enclosed ( ) Check/Money Order ( ) 
BDl Me ( ) 



RECENT PUBLICATIONS AND PRESENTATIONS 
BY TEACHING RESEARCH STAFF 

Articles 

Bullis, M., & Reiman, J. (1992). Development and 
preliminary psychometric properties of the Transition 
Competence Battery for deaf adolescents and young adults. 
Exceptional Children. 59, 12-26. 

Fredericks, B. (1992). The right environment for a behavior 
program. Meaningful Lives, ^(2), 8-9. 

Fredericks, B. (1992). Staff - an essential ingredient in 
functional assessment. Meaningful Lives, ]J3), 8-9. 
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